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Description 



Digitization of work processes using 
wearable wireless devices capable of 
vocal command recognition in noisy 
environments 

Background of Invention 

[0001] The present invention relates to data acquisition and con- 
trol of machinery via a system and apparatus for digitiza- 
tion of work processes over wireless communications net- 
work using wearable computation/ communication de- 
vices having vocal command recognition in noisy environ- 
ments. 

[0002] Acquisition of accurate and up-to-date information con- 
cerning the operational status, condition and ongoing 
performance of all equipment that plays a critical role in a 
particular industrial process or system such as, for exam- 
ple, the large electric motors used in industrial manufac- 
turing processes or turbines used in commercial power 



generation and the like is crucial in assuring and main- 
taining a successful commercial operation. Consequently, 
considerable efforts have been expended to develop and 
improve upon the existing conventional methods and ap- 
paratus used for monitoring and maintaining the opera- 
tion of such "critical" equipment. In this regard, robust 
methods of inspection and fault prediction are highly de- 
sirable for equipment used in commercial power genera- 
tion systems, since whenever such equipment must be 
taken off-line for repair or maintenance its inoperability 
may adversely impact other revenue generating processes 
such as commercial production and manufacturing facili- 
ties. 

[0003] In order to detect impending failures, robust procedures 
for equipment inspection or predictive maintenance usu- 
ally involve the monitoring and inspection of a variety of 
equipment operational parameters over a period of time. 
Conventionally, one or more parameters of a critical piece 
of machinery or equipment are monitored over time and 
data is collected at predetermined intervals. A mainte- 
nance recommendation is then triggered whenever the 
value of a particular monitored parameter happens to be 
recognized as exceeding a predetermined threshold. 



[0004] One contemporary technological trend is to automate 
such inspection and monitoring processes as much as 
possible by affixing sensors and transducers to critical 
equipment and continuously collecting and monitoring 
important operational parameters through a variety of on- 
line and off-line electronic data monitoring techniques. In 
this manner, parameter data for a system or process may 
be continually tracked and an alarm may be immediately 
triggered if threshold values for particular parameters are 
detected as being exceeded. However, some level of hu- 
man intervention and interaction is almost always still re- 
quired especially in the case of large or complex industrial 
systems such as, for example, land-based power genera- 
tion systems. 

[0005] At many land-based power generation plants, the com- 
pressor components of large gas turbines engines used in 
power generation must be operated at mechanically 
stressful high pressures to be efficient and, consequently, 
numerous tests are often performed on-site by a field 
maintenance engineer to ensure that such engine compo- 
nents are maintained in prime condition. Such tests typi- 
cally rely on a variety of different instruments and the 
field maintenance engineer may first need to attach sen- 



sors at several different locations in/on the turbine to 
measure specific engine parameters. The measured pa- 
rameter values are then recorded and compared by the 
field engineer to known acceptable values to determine 
whether the gas turbine should remain in operation. 
[0006] If certain measured parameter values are determined to 
be outside of an acceptable range, the field engineer may 
need to communicate with a remotely located operations 
control engineer who is responsible for controlling the 
immediate operation of various components such as 
valves, pumps, switches, etc. of the turbine. As such, field 
testing procedures, inspections and even routine opera- 
tional activities can quickly become very labor intensive, 
requiring multiple engineers to perform even the most 
simple tasks. Consequently, in order to provide a working 
line of communication between all involved personnel, ra- 
dio and/or telephone communication links must be estab- 
lished and constantly maintained thus complicating and 
slowing down the overall inspection/testing process. In 
addition, access and communication with remote database 
and/or needed computational facilities can often be slow 
or non-achievable in real-time, thus adding further ineffi- 
ciencies to ongoing inspection and predictive maintenance 



procedures. 

[0007] In many commercial environments, conventional tele- 
phone line or cable connections are often used to estab- 
lish corporate intranet connections or virtual private net- 
works (VPN) for accessing various in-house computational 
facilities and resources. Unfortunately, at a remote point 
of service in the field, the loss of information that is often 
experienced with conventional telephone and cable line 
communications does not allow, for example, an immedi- 
ate use of remote computational facilities and various 
utility software applications or archived data that may be 
stored on a remote server on the network. Often, test/ 
inspection data that is obtained and stored at a testing 
site in the field must be up-loaded off-line to a remote 
server after completing the testing/inspection process. In 
addition, since many test and inspection procedures must 
be performed by several engineers who communicate 
contemporaneously, the large volumes of data often cause 
issues with the bandwidth limitations of conventional 
telephone and radio communications. Furthermore, in- 
stallation costs and the time required for setting up the 
custom land-line communication links needed for such 
activities is often very high. Thus, there is a need to over- 



come the many communications problems and inefficien- 
cies tliat are often encountered during tlie course of im- 
plementing robust test and inspection procedures, pre- 
dictive maintenance and/or repair sclieduling of "critical" 
industrial equipment and systems, especially with respect 
to equipment and machinery at power generation plants. 
[0008] The problem with many of the inspections performed at 
power generation plants are due to the use of multiple 
pieces of test equipment and associated data entry tasks. 
Multiple inspectors are required to collect, analyze and 
record data from a single testing activity. More efficient 
means are required to reduce cost and the potential for 
human error. One proposed solution to at least some of 
the above problems may be provided through the use of a 
portable/wearable computer/data entry device operated 
by a field engineer that is coupled to a network of remote 
computational and data storage devices by using high 
bandwidth connectivity via a combination of satellite and/ 
or high bandwidth landline connections and an on-site 
wireless local area network (LAN) system. Such an ar- 
rangement allows a digitization of work processes by a 
single person which typically may have required multiple 
personnel to complete. One example of such a system is 



provided in commonly assigned copending U.S. Patent 
Application Ser. No. 09/844,270, filed April 30, 2001, en- 
titled "DIGITIZATION OF WORK PROCESSES THROUGH THE 
USE OF A WIRELESS NETWORK WITH USER WEARABLE END 
DEVICES", which is hereby incorporated by reference 
herein. 

[0009] Unfortunately, the portable computer/data entry devices 
of the above mentioned system often require numerous 
and extensive manual command and data entry operations 
to acquire, analyze and manipulate data for complex work 
processes or, for example, to input multiple commands to 
conduct specific plant operations and testing procedures 
or to perform other types of communication tasks. Per- 
forming such manual entry operations can be both te- 
dious and time consuming, as well as being prone to er- 
ror. In addition, manual entry of commands and data de- 
tracts from a field engineer's attention and efficiency at 
the work site and results in reduced productivity and in- 
creased error. One solution to this problem, that would 
potentially increase a worker's productivity while minimiz- 
ing data and communication entry errors, may be 
achieved through the implementation of conventional 
voice recognition/enabling technology. However, conven- 



tional voice recognition/enabling technologies are much 
too susceptible to errors induced due to the extensive 
background noise that occurs in a typical power plant en- 
vironment. Consequently, a voice recognition/enabling 
technology which is operable in noisy environments and is 
implementable in a work process digitization system us- 
ing portable/wearable computer/data entry type devices 
would be highly desirable. 
Summary of Invention 

[0010] The present invention relates to a method and apparatus 
that enhances the productivity of field engineers through 
a combination of high-bandwidth wide area network 
(WAN) connectivity, on-site wireless local area network 
(LAN) systems, and wearable wireless computer devices 
having vocal command recognition and ambient noise 
suppression. (For the purpose of the present discussion, a 
"wearable" device encompasses computer and communi- 
cations devices that are generally mobile and adapted to 
be carried or worn by a user.) In particular, through the 
use of a hands-free vocal-command recognition interface 
that can operate reliably in noisy industrial environments, 
the present invention enables a field engineer/inspector 
to remotely control and individually operate complex 



equipment and processes (e.g., gas turbine and generator 
equipment in a power generation plant) wliile also having 
the capacity to roam throughout the physical premises as 
opposed to being limited to the fixed location of an 
equipment control room. For example, using the present 
invention a field engineer may inspect, test and/or control 
various operations of complex machinery and processes 
from virtually any location in the plant. 
1] The combination of using a hands-free vocal-command 
driven communication device and high bandwidth wireless 
connectivity with multiple people and computer network 
resources provides an enhanced degree of freedom and 
control in the performance of complex work processes by 
a single or few persons where, in the past, such work pro- 
cesses have required multiple personnel to complete. Us- 
ing the present invention, a power plant field engineer 
may roam around or remain mobile at the location of spe- 
cific equipment while performing multiple tasks, such as 
simultaneously controlling the operations of the equip- 
ment while acquiring parametric data, by simply vocaliz- 
ing specific commands. In this manner, the present inven- 
tion reduces or eliminates the inefficiency and inconve- 
nience of having to relay information and commands to 



other operations control personnel stationed in a control 
room/station which is typically located some distance 
away from the actual equipment or processes being con- 
trolled. Moreover, the field engineer may use the present 
invention to leverage the expertise of a remotely located 
engineer/expert by providing that person with real-time 
data, video, and control connectivity in order to resolve a 
problem in a collaborative manner. 
[0012] Unfortunately, the work environment of a typical electrical 
power generation plant has a variety of noisy equipment 
that may produce as much as 70-100 dB of baseline am- 
bient background noise. This noise may be characterized 
as comprising both a "white" noise component created, 
for example, by operating turbines, and a "colored" (i.e., 
frequency-dependent) noise component contributed by 
related maintenance activities and the intermittent opera- 
tion of various plant equipment such as cranes and weld- 
ing machines. Voice recognition in such noisy environ- 
ments requires continuous and nearly real-time mitigation 
of the ambient background noise. The present invention 
accomplishes this through the use of an adaptive noise 
signature recognition and canceling process implemented 
by the communication/computer device and a directional 



microphone for optimal reception of the speal<er's voice. 
An integral ambient noise suppression process is provided 
for a voice-responsive computer-communication device 
for use in highly noisy industrial environments. This am- 
bient noise suppression processing significantly improves 
real-time voice recognition of spoken commands despite 
the presence of a high degree of ambient background 
noise. In particular, the integral noise suppression process 
of the present invention may adaptively react in real time 
to various and changing ambient noises associated with 
different environments and effectively operates to elimi- 
nate such background noises when detected in an audio 
signal to improve voice recognition. 
[0013] As presented in at least one example embodiment of the 
present invention disclosed herein, the wearable com- 
puter/communications device is programmed to imple- 
ment an adaptive noise cancellation process that has a 
great robustness to changes in ambient non-speech 
background noise. This is accomplished at least in part by 
programming the device to detect the existence of prede- 
termined noise signatures in the frequency domain of a 
pulse code modulated signal from a microphone associ- 
ated with the wearable computer/communications device 



and subtracting those noise signal signatures from the 
microphone signal. Active filtering is also used to elimi- 
nate or significantly reduce both white and colored back- 
ground noise. Once the non-speech ambient background 
noise is removed from the microphone signal, a conven- 
tional speech recognition technology, commercially avail- 
able through Conversational Computing Corporation 

TM 

(Redmond, Washington) as "Conversay Advanced Sym- 
bolic Speech Interface" (CASSI), is employed to provide 
speaker independent voice-recognition. For the example 
environment presented herein, a work process involves 
the need for controlling a gas turbine in a power genera- 
tion plant while measuring turbine and generator parame- 
ters from a location at or near the turbine. The power 
plant control arrangement includes a primary computer/ 
processor system comprising an equipment controller at a 
fixed location within the plant. This local processor sys- 
tem receives operational data from all critical machinery 
and equipment in the power plant. In the example system 
presented for providing digitization of work processes, 
the local processor system includes a land-line based lo- 
cal area network (LAN) that is provided with at least one 
wireless device LAN access interface station at a fixed lo- 



cation within tlie power plant. This LAN access interface 
station allows a mobile wireless computer/commu- 
nications device which is carried by a field engineer to es- 
tablish and maintain communication with the plant equip- 
ment controller on the LAN. The equipment controller is 
also capable of receiving commands from the voice- 
responsive wireless computer/communication device that 
is carried by the field engineer via the wireless communi- 
cations interface/LAN access station to control the gas 
turbine. 

[0014] In addition a fixed wireless communications access station 
or bridge antenna station having a transceiver system may 
be employed for transmitting and receiving signals from 
either the voice-responsive wireless computer/com- 
munication device or the wireless communications device 
LAN access station to a remote private network computer 
server or facilities which, for example, may in turn be 
communicatively coupled to the wireless access station to 
provide a wider area communication network arrange- 
ment. Such an arrangement may include another WAN/ 
LAN, a satellite communications link, the public Internet 
backbone. The private server/computer facilities may in- 
clude, for example, security firewall arrangements, addi- 



tional computational equipment and/or a database for 
storing application data that is also all made accessible to 
the field engineer via the network communications ar- 
rangement. 

[0015] A particularly beneficial aspect of the present system, is 
its ability to provide high bandwidth connectivity to a field 
engineer to enable, for example, the use of web-portal 
applications stored in a remote server at the point of ser- 
vice of the field engineer. Immediate access to web-portal 
applications is also made possible for a field engineer sit- 
uated at an equipment installation site that does not oth- 
erwise have access to a landline telephone link via two- 
way satellite connectivity. A further beneficial aspect of 
the present system arrangement is that it allows a field 
engineer that is monitoring a piece of equipment at a first 
location share information with another person/engineer 
monitoring similar equipment at a different remote loca- 
tion such that the collaborative effort may be used to fine 
tune one piece of equipment based on parameters or set- 
tings of the equivalent equipment located at the other 
site. 

Brief Description of Drawings 
[0016] jhe features of the invention believed to be novel are set 



forth with particularity in the appended claims. The inven- 
tion itself, however, both as to organization and method 
of operation, together with further aspects and advan- 
tages thereof, may best be understood by reference to the 
detailed description taken in conjunction with the follow- 
ing drawings, in which: 

[0017] FIGURE 1 is a schematic diagram illustrating an example 
system for implementing the digitization of work pro- 
cesses using a wireless voice-responsive wearable com- 
puter/communication device; and 

[0018] FIGURE 2 is a high-level process flow diagram illustrating 
an example process implemented by a voice-responsive 
computer/communication device to provide adaptive am- 
bient background noise cancellation for improved speech 
recognition in noisy environments. 
Detailed Description 

[0019] Referring first to FIGURE 1, there is shown a schematic di- 
agram illustrating an example system for implementing a 
digitization of work processes such as the inspection, 
testing and control of power plant machinery using one or 
more wireless voice-responsive wearable computer/ 
communication devices. A conventional power plant, gen- 
erally indicated at 10, typically includes at least a gas tur- 



bine engine 11, a generator 12 and possibly other ma- 
chinery and equipment that may be considered as "criti- 
cal" to the production of electrical power. In this example, 
the critical machinery and equipment in power plant 10 
are fitted with sensors and operational parameter moni- 
toring devices that are communicatively coupled either di- 
rectly or via local wireless links to an on-site, or relatively 
local, processor and operations control system 
(Equipment Controller 13). Such a control system may also 
include a local human-machine interface (HMI) device for 
allowing a technician or engineer to operate and control 
the operations of the power plant. 
[0020] A wearable wireless mobile computer/communication de- 
vice 16 is carried around by a technician or engineer 17 
while conducting tests or inspection tasks. Conventional 

TM 

application software such as, for example, Netmeeting 
available from Microsoft Corporation, may be used to 
communicate data between processor system 13 and 
wearable wireless mobile computer/communication device 
16 carried around by a user 17. Wearable computer/ com- 
munication device 16 comprises a conventional mobile 
computer type device that may be easily carried or worn 
by field personnel and used for data entry or display. 



Computer/communication device 16 also includes con- 
ventional means for implementing wireless communica- 
tion of data (e.g., wireless LAN interfacing hardware). In 
addition, computer/communication device 16 is provided 
with appropriate hardware and software for implementing 
voice recognition and responding to vocalized commands. 
In particular, computer/communication device 16 is pro- 
vided with an adaptive background noise suppression ca- 
pability and a directional microphone which enable the 
device to perform accurate voice recognition even in very 
noisy industrial environments. 
[0021] Operational parameter data from sensors on gas turbine 
11 and/or generator 12 may be received by the processor 
system 13 either through direct electrical connection or 
via a conventional local wireless communications network 
or link. The processor system 13 may include a database 
system for storing operational data and forwarding the 
stored data to a wireless communications device interface 
such as, for example, fixed point wireless access station 
14. Operational data received by the interface unit 14 is 
communicated to wearable computer/communication de- 
vice 16 carried by a mobile user 17. The processor system 
13 may include a controller for receiving instructions from 



field engineer/mobile user 17 carrying the wearable com- 
puter/communication device 16 to vary and control oper- 
ational parameters of gas turbine 11 and/or generator 12. 
[0022] The wearable computer/communication device 16 used by 
field engineer/mobile user 17 may also be communica- 
tively linked to private network computer facilities 15 such 
as, for example, a corporate server system or the like, via 
a more extensive communications network generally indi- 
cated at 18. To ensure end-to-end security, all communi- 
cations over the network may also be authenticated with a 
central server that employs a periodically rotating encryp- 
tion key. 

[0023] Referring now to Figure 2, a high-level process flow dia- 
gram is illustrated which details an example process that 
may be implemented by a voice-responsive computer/ 
communication device to provide both active noise reduc- 
tion and adaptive ambient background noise cancellation 
for improved speech recognition in noisy environments. 
As indicated at block 20, a computer/communication de- 
vice is configured to implement the improved noise can- 
cellation process of the present invention when running a 
software application or conducting a process that requires 
the input of extraneous data or commands. Such a pro- 



cess may be a conventional piece of software such as Ex- 
ceT"^ or Access"^"^ or a spreadsheet application that re- 
quires the entering of data into a form, or it may be some 
other user specific software application that provides, for 
example, voiced commands/instructions to a remote ma- 
chine controller for the purpose of remotely controlling 
the operation of a particular piece of equipment in real 
time. During use of the computer/communication device, 
and particularly whenever a user speaks and directs his/ 
her voice toward the device, the analog sound waves of 
the user's voice are received by a directionally sensitive 
microphone connected to the computer /communication 
device, as indicated in block 21. Preferably, a high-quality 
microphone (not shown in the FIGURES) is used which has 
a specific directional field of operation outside of which 
the reception of sounds are diminished. The microphone 
may be integral or external to computer/communication 
device 16 and is positioned or oriented during use of the 
device so as to optimize reception of a user's voice and 
minimize reception of ambient sounds. 
[0024] In a preferred example embodiment of the present inven- 
tion, ambient background noise cancellation of sounds 
received by the microphone when a user speaks is per- 



formed in stages: First, an active elimination or reduction 
of ambient bacl<ground noise is performed on tlie audio 
signal produced by the microphone in the analog domain 
using conventional analog filtering techniques. As indi- 
cated in block 22, this active noise cancellation reduces or 
eliminates certain predetermined "colored" (e.g., fre- 
quency band specific) background noises as well as ambi- 
ent background noise generally. In addition, the receiver 
directionality properties of the directional microphone are 
capitalized upon when capturing the user's voice such di- 
rectionality minimizes background noise and can improve 
the signal-to-noise ratio of the microphone signal by a 
factor of two or more. Next, the analog microphone signal 
is then sampled and pulse code-modulated. Adaptive 
noise cancellation is then performed in the digital domain, 
as indicated at block 23. In this stage, the digital pulse 
code modulated microphone signal is transformed into 
the frequency domain and predetermined ambient noises 
and/or noise bands are continually identified and sub- 
tracted according to their characteristic digital frequency 
domain signature. The computer/communication device 
may be readily programmed to identify one or more dif- 
ferent noise signatures. The identification of such noise 



signatures domain may be based on known noise signa- 
tures and/or customized according to specific noise sig- 
natures at a particular location. In this manner, the back- 
ground noise elimination process of the present invention 
is responsive to different types of background noise and 
is highly adaptive to any changes in the ambient back- 
ground noise. This multi-stage noise elimination process 
provides improved recognition of voiced commands and/ 
or spoken information directed toward the device during 
operation in noisy environments. 
[0025] In a following stage, as indicated at block 24, speech- 
specific noise elimination is performed using conventional 
statistical noise cancellation processes such as Cepstral 
domain subtraction/normalization. For example, a com- 
mercially available software speech recognition engine, 

TM 

such as Conversay by Conversational Computing Cor- 
poration (Redmond, Washington), may be used for provid- 
ing both speech-specific noise elimination algorithms and 
performing speech recognition. In this manner the device 
is rendered capable of providing speaker-independent 
voice-recognition. Next, as indicated in block 25, the rec- 
ognized commands and/or data are entered into the cur- 
rent application. During this process, as indicated in block 



26, two-way interactive communication is maintained be- 
tween mobile computer/communication device 16 and 
power plant processor system/equipment controller 13 
(FIGURE 1) and/or private network/computer facilities 15 
via communications network 18. 
[0026] While the invention has been described in connection with 
what is presently considered to be the most practical and 
preferred embodiment, it is to be understood that the in- 
vention is not to be limited to the disclosed embodiment, 
but on the contrary, is intended to cover various modifica- 
tions and equivalent arrangements included within the 
spirit and scope of the appended claims. 



